This section is intended to introduce to the reader various aspects of art, which may be related to various aspects of the present disclosure that are described and/or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present disclosure. Accordingly, it should be understood that these statements are to be read in this light, and not as admissions of prior art.
In the framework of stereo (or multiple views) imaging, stereo (or multiple views) video contents need to be created, processed and reproduced on a 3D capable screen. Processing of stereo video contents allows the creation or enhancement of 3D information (for example disparity estimation). It also allows the enhancement of 2D images using 3D information (for example view interpolation).
Normally, stereo video contents are created from two (or more) captured 2D videos. 2D videos are videos in a classical sense having a temporal series of frames, each being a classical 2D image consisting of lines of pixels. Each pixel has a color, defined by color coordinates in a color space. Generally, a stereo imaging is multiple view imaging where more than two 2D videos are captured from the scene. Stereo imaging can also be generated in a synthetic manner from 3D models and computer graphics. In this way, animation can be created resulting in stereoscopic images and other related images having multiple views.
FIG. 1 shows a conventional process for creating multiple views. Several conventional 2D cameras 22 are shown which are fixed onto a rig output for creating raw image views 33. In one embodiment, a camera calibration module 44 performs camera calibration. This calibration provides an estimation of external and internal camera parameters based on the captured raw image views 33. Subsequently, the view generator module 55 generates calibrated views 66.
Each camera is defined by external and internal parameters. External parameters are expressed in the world-coordinate system and define the position and orientation of a camera. External parameters include a 3D rotation matrix R and a 3D translation vector t. Internal camera parameters include the focal length f and the coordinates px,py of the principal point in the image plane.
For example, for a pin hole camera, a point in homogeneous scene-world-coordinates X=(X Y Z 1)T is projected onto an image position x=(x y)7 according tox=PX with P being the camera matrix according toP=K[R/Rt]with R being a 3×3 rotation matrix, t being a 3D translation vector and with K being the camera calibration matrix defined according to
      K    =                  (                                            f                                                                                                          p                                                                                                                              f                                      p                                                                                                                                                                                                  1                                      )            ⁢                          ⁢      with      ⁢                          ⁢      2      ⁢                          ⁢      parameters        or            K      =                        (                                                    f                                                                                                                                            p                  x                                                                                                                                                                  f                                                              p                  y                                                                                                                                                                                                                                                1                                              )                ⁢                                  ⁢        with        ⁢                                  ⁢        3        ⁢                                  ⁢        parameters              ,                  ⁢    or        K    =                  (                                            fx                                                                                                                          p                x                                                                                                                                            fy                                                      p                y                                                                                                                                                                                                                1                                      )            ⁢                          ⁢      with      ⁢                          ⁢      4      ⁢                          ⁢              parameters        .            
Camera calibration can be carried out, for example, using an object for calibration with known feature points at positions (fX,i fY,i fZ,i 1)T in homogeneous scene-world-coordinates and their projections into the image plane at positions with homogeneous coordinates (fx,i fy,i 1)T. The camera parameters can be estimated according to the method described by Richard Hartley and Andrew Zissermann in their book entitled “Multiple view geometry in computer vision” published at Cambridge press on Mar. 24, 2004.
The next step of known processing in multiple view imaging is view rectification. Rectification is a necessary step to transform the views (two or more views) taken by cameras on a stereo rig to the views of which geometries are corrected. By performing rectification of the views, correcting the geometry of the views so that each view appears to be taken by cameras having a same image plane. However there are some limitations on the rectification. For example, vertical disparity cannot be corrected easily without a rotating the views, which is in most cases unacceptable. Vertical disparity appears for example in case the optical centers of the cameras are not horizontally aligned, i.e. have different vertical positions.
This problem related to vertical disparity is explained for example by Simon Reeve et al. in his presentation “Basic principles of stereoscopic 3D” as a white paper from BskyB on 2010. Vertical disparity is caused by cameras that do not have the same position or viewing direction in vertical direction. Even if rigs of stereo cameras are mechanically calibrated, a residual vertical misalignment often remains. Vertical misalignment creates problems in viewing stereo content.